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ABSTRACT 

In Pioneer Institute’s November 1995 "Dialogue,” nine 
school-choice advocates critiqued a draft manuscript of "Who Chooses, 
Who Loses? Culture, Institutions, and the Unequal Effects of School 
Choice," edited by Harvard professors Richard Elmore, Gary Orfield, 
and Bruc^ Fuller. This publication contains the response of Professor 
Fuller to the dialogue, along with a reaction by Terry M. Moe, senior 
fellow at the Hoover Institution, Stanford University. Fuller argues 
that the Pioneer Institute invited only proponents of choice to join 
the conversation, that they disregarded the book's empirical 
findings, and that the programs described in the book were chosen on 
the basis of empirically sound evaluation. Moe asserts that the 
Harvard volume is not a true study, that it emphasizes the negative 
aspects of choice, and that it is biased against school choice. He 
argues that problems of inequality can be resolved through better 
program design. Two figures are included. An inset contains the 
response of Professor Paul Peterson to remarks made by Professor 
Fuller about Peterson's "Wall Street Journal" piece, in which 
Peterson praised a San Antonio voucher program. Peterson argues that 
Fuller's concern for selection effects in San Antonio contrast with 
his (Fuller's) disregard for selection effects in the Milwaukee 
program. (LMl) 
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Two Perspectives on the 
Continuing Debate Over 
School Choice 




In our November 1995 Dialogue,, “Responses to a Harvard Study on School 
Choice: Is It a Study at All ?” we gathered nine school choice experts to critique a 
draft manuscript of Who Chooses, Who Loses? Culture, Institutions, and the 
Unequal Effects of School Choice, edited by Harvard Professors Richard Elmore , 
Gary Orfield, and Bruce Fuller. Upon receiving the Dialogue, Professor Fuller 
requested the opportunity to respond . In the following pages , Pioneer presents 
his response , along with a reaction by Terry M. Moe , senior fellow at the Hoover 
Institution , Stanford University. 
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Please, Can We Focus on Evidence? 

By Bruce Fuller 

Pioneer Institute’s November 1995 Dialogue was dedicated to hearing from crit- 
ics of Harvard’s recent review of school choice programs. Our research group appre- 
ciates Pioneer’s effort to stimulate civic debate on these new evaluation findings. 

But the Dialogue was odd on two scores. First, only proponents of choice 
were invited to join the conversation. Nine advocates were invited to comment 
on our three-year research' project. No critics of school choice were invited to 
join this chat among the already converted. Second, the Harvard project’s central 
purpose was to review the most solid empirical work we could find, focusing on 
the local effects of variably constructed choice programs. We shared the entire 
book manuscript, containing ample empirical evidence. We held the (apparently 
idealistic) expectation that the Pioneer staff and their chosen commentators would 
read the empirical studies, not simply respond to the media summaries they so 
vociferously attacked. Sadly, many in Pioneer’s corral of critics opted to simply 
raise the decibel level of the rhetoric, rather than address the evidence. 

In our unrequited hope that the quality of debate can be raised, I want to 
recap the aims of the Harvard project and our core findings. The complete book 
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is available from Teachers College Press, Columbia 
University, and I urge you to study the empirical find- 
ings. I also want to thank Charlie Glenn, Terry Moe, 
Diane Ravitch, Alfred Taubman, and others who have 
contacted us to more carefully talk through our find- 
ings — with the collective objective of improving how 
choice initiatives are structured in the future. 

Background 

In 1991, my colleagues Richard Elmore and Gary 
Orfield joined with me to invite researchers active in the 
school choice field to present their empirical findings at 
a seminar series. Our aim throughout has been to ex- 
amine fine evaluation evidence on three specific issues: 

• Which parents respond enthusiastically to choice 
options when they become available via magnet 
schools, voucher programs, or charter schools? 

• Under liberalized market conditions, do inventive 
new forms of schooling arise, and which key ac- 
tors help stimulate organizational innovation? 

• What effects of choice schools are empirically ob- 
served, specifically in terms of parental satisfac- 
tion with new “choice schools” and in terms of 
student achievement gains? 

These questions are central to the arguments ad- 
vanced by choice proponents, and contested by oppo- 
nents. They are questions that can be informed by 
empirical evidence. Our principal concern is not with 
issues of political philosophy or important normative 
questions around how to balance individual parent 
interests and the ever-slippery public interest. These 
are not settled by positivist findings. Our team’s single 
aim was to assess available facts on the three empiri- 
cally assessable questions. 

Key Empirical Findings 

Richard Elmore and I presented our findings at 
the annual meeting of the National Conference of State 
Legislatures, held last summer in Milwaukee. Three 
key findings proved to be most controversial. We 
showed how choice programs are quite popular, espe- 
cially among working class and inner city parents who 
face low quality neighborhood schools. Despite this 
enthusiasm, however, no evidence is available to sub- 
stantiate that children moving into “choice schools” 



are learning at a higher rate than those who remain in 
neighborhood schools. And inequities can result from 
choice programs, where parents already highly in- 
volved in their children’s schooling (attending school 
meetings, checking their homework, for example) 
make better shoppers. Left behind in lousy neighbor- 
hood schools are children whose often impoverished 
parents may not have time or wherewithal to press 
their children to do well in school. 



programs are quite popular, 
especially among working class and inner 
city parents who face low quality neigh- 
borhood schools. Despite this enthusi- 
asm, however, no evidence is available 
to substantiate that children moving into 
“choice schools ” are learning at a higher 
rate than those who remain in neighbor- 
hood schools. 



A fourth finding, receiving less attention, is that 
the ability of local activists and educators to create 
exciting new schools depends heavily on prior levels 
of community organizing, the structure of the choice 
program, and levels of public subsidy. This is the miss- 
ing link which explains why parents’ enthusiastic re- 
sponse to choice does not automatically yield change 
inside classrooms where promised learning gains are 
not being observed. No one in the private sector ar- 
gues that market conditions magically create effective 
firms. All sorts of internal organizational factors come 
into play. We have paid scant attention to the issue of 
how competitive pressures will lead to the crafting of 
exciting new schools. 

A Flood of Criticism 

Following public release of our findings we have 
been attacked in a number of ways. Our critics claim 
that the media emphasized the negative side of our 
findings. Or that this was not really a study (in Howard 
Fuller’s words). Or that the authors are biased and 
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have a record of being anti-choice. Or that we selected 
choice programs with problems, not the “right pro- 
grams,” which are more effective. One critic in the 
Pioneer newsletter, Mary Anne Raywid, claimed our 
work does not meet academic standards of rigor. These 
are serious charges. 

I must admit to being surprised by a portion of 
these slams, especially coming from choice advocates 
in university settings — critics who should abide by 
rules of evidence and logical argument. Indeed, I was 
hoping that critics would focus on the quality or limi- 
tations of the evidence . For example, we have repeat- 
edly emphasized how much more evidence is needed 
before we can gain a clear understanding of how pro- 
gram features interact with local demographic and 
economic conditions, to yield variable effects. 

But the rather low-brow vein of criticism, follow- 
ing release of our basic findings, has been more com- 
mon than careful assessments of the evidence put for- 
ward. Civic debate over school choice and the future 
role of government in our society is of crucial impor- 
tance. Scholars in particular should provide informed 
leadership, not reproduce unsubstantiated claims and 
good intentions not backed by evidence. We should 
relentlessly demand of politicians and activists that 
they produce evidence that backs their claims. When 
university-based advocates compromise this funda- 
mental tenet of the academy, they contribute nothing 
new and simply undercut the public’s hope that the 
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university will enrich public discourse. Parenthetically, 
we should be just as tough on equally faith-filled ad- 
vocates of “systemic school reform,” or networks like 
the Coalition for Essential Schools. These opponents 
of choice genuflect just as frequently, as they pray that 
bureaucratic-driven reform will work. 



Jl arents already highly involved in their 
children’s schooling ( attending school 
meetings, checking their homework, for 
example) make better shoppers. Left be- 
hind in lousy neighborhood schools are 
children whose often impoverished par- 
ents may not have time or wherewithal to 
press their children to do well in school. 



Political Rights vs. Empirical Effects 

Choice starts with the philosophical question of 
whether a parent should have the political right to 
leave the neighborhood school and search for a better 
alternative. My own belief is that in a democratic so- 
ciety it is difficult to argue against this “right.” But no 
part of our book takes a position on the normative 
question of whether and how this aspect of political 
authority should be altered. Democratic societies con- 
stantly struggle with how to properly balance the in- 
dividualistic desires of citizens against a broader shared 
public interest. 

Can Inequality Get Worse? 

Some critics in the Pioneer Dialogue argued that 
the degree of inequality and segregation in American 
schools is so bad, we must try something new. This is 
not reassuring. If choice proponents are concerned 
with the likelihood of unequal results, let’s work to 
honestly evaluate choice experiments: to assess the 
magnitude of the problem and figure out ways to mini- 
mize unfair effects. Joe Nathan and others point to 
the Cambridge “universal choice” option, where all 
parents express their top three preferred schools. But 
note that in Cambridge, public authority remains and 
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squarely in the picture to balance individual family 
preferences against the local community’s shared in- 
terest in avoiding overcrowding at popular schools 
and in ensuring ethnic diversity within each school. 

Local Effects 

Please read the evidence reported in our book. 
Our critics are quick to read chapter titles and see 
which cities were included in the Harvard effort, in- 
cluding Detroit, Milwaukee, Montgomery County, and 
San Antonio. But these critics rarely work their way 
through the empirical results. I much prefer Terry 
Moe’s thoughtful analysis and informed attack of our 
evidence — both the quality of what we review and 
what evidence does not yet exist. In the original Pio- 
neer critique, Mr. Moe pointed out that test scores are 
not a complete measure of desired outcomes: “Many 
families are looking for a better moral climate, stron- 
ger discipline, safer schools, and more attention for 
their children. These things do not necessarily show 
up in test scores.” Indeed, Milwaukee and San Anto- 
nio demonstrate how many black and Hispanic par- 
ents seek schools that have curricula focused on their 
own cultural heritage, language, and social mores. 



Bravo! Mr. Moe’s critique opens up debate on how 
we can best assess the forms of learning that parents 
themselves value. We all learn from this level of debate. 

Picking the “Right” Choice Programs 

Our project can be fairly criticized for including 
an insufficient range of choice programs. We did our 
best to search out empirically sound evaluation stud- 
ies, both qualitative and quantitative research. But we 
are constrained by the paucity of sound evaluations. 
Some experiments, like charter schools, are recent in 
origin. Equally troubling, however, is when loose as- 
sessments are bandied about as hard evidence. Reports 
coming from business financed groups in Milwaukee, 
for instance, attribute all sorts of student effects to 
their experience in choice schools, without even think- 
ing about how to first control on the independent ef- 
fects of family background factors. Harvard Profes- 
sor Paul Peterson recently celebrated San Antonio’s 
alleged achievement effects in The Wall Street Jour- 
nal , without even acknowledging the sharp selection 
effect that confounds these findings. Pro-choice inter- 
est groups — if seriously concerned about the positive 
and negative effects of their programs — should work 
with technically able researchers. Hiring lobbyists to 
conduct “evaluations” undercuts the legitimacy of their 
own cause. 

Risking Alienation 

The scornful and imprecise character of a portion 
of our critics is personally distressing to me. I think 
you run the risk of alienating both political moder- 
ates and empirical scholars who heretofore have been 
agnostic toward school and family choice. Over the 
past five years, I have been studying the dynamics of 
the mixed preschool market. The sector is far ahead 
of K-12 education in terms of bringing together vari- 
ous sources of revenue — parental fees, tax credits, 
vouchers, and direct subsidies to preschools — to yield 
a diverse and colorful range of locally controlled or- 
ganizations. My work asks about the benefits and 
possible inequities resulting from this pro-choice in- 
stitutional arrangement. When asked to summarize a 
series of empirical studies, I argue that when subsi- 
dies are targeted to poor and working class families 
and when minimal regulation of quality is in place, 



ERjt 



6 



5 



^Choice starts with the philosophical 
question of whether a parent should have 
the political right to leave the neighbor- 
hood school and search for a better al- 
ternative. My own belief is that in a 
democratic society it is difficult to argue 
against this “right. ” 



family access to preschools is fairly equal and quality 
is quite evenly distributed across diverse communities 
within many states. 

Despite this long line of work and my empirical 
arrival at a point that specifies when and how pro- 
choice policies can be largely effective, my critics ironi- 
cally claim that I am “anti-choice” and “biased.” It is 



the fact that my critics are uninformed that I find dis- 
appointing. It makes me feel that I must be extremely 
careful before uttering anything that might sound pro- 
choice, since advocates will take little care to read my 
findings thoughtfully or with much attention to fine 
points. It says to me that some members of the aca- 
demic community have such a fervent political agenda 
that empirical findings make little difference in ad- 
justing their thinking. 

No member of our research team is naive about 
the intensity of the polemics surrounding these issues. 
The choice question evokes larger than usual doses of 
argumentation and venom. The debate is enmeshed 
in how the role of government is recrafted and how 
we balance a brittle public interest against the indi- 
vidualistic instincts of some American families who 
are rightfully worried about how their youngsters will 
get ahead. The localized interests of ethnic groups 
continue to challenge how we define the public inter- 
est, ranging from affluent whites looking for safety 



In his response , Professor Fuller mentions a 
Wall Street Journal piece by Professor Paul Peter- 
son, in which Peterson praises a voucher program 
in San Antonio . Fuller claims Professor Peterson 
did not consider all the relevant data before tout- 
ing the program's impact on student achievement. 
Peterson has asked for an opportunity to respond 
to Professor Fuller's comments . 

Mr. Fuller would do well to read my Wall Street 
Journal piece carefully. In the paragraph immedi- 
ately following the one Fuller quotes, I say, “These 
findings are less than definitive... But if taken at 
face value , they suggest that voucher programs, in 
order to be effective for low-income, inner-city 
families, need to include religious schools.” 

Fuller’s concern for selection effects in San An- 
tonio contrasts sharply with his disregard for se- 
lection effects in the state-funded choice program 
in Milwaukee. In his news release of July 13, 1995, 
“Studies Show School Choice Widens Inequality,” 
he quotes Richard Elmore as saying “Thousands 
of children have participated in Milwaukee’s pub- 
lic-private voucher experiment over the past three 



years, yet we see no discernible gains in learning.” 
Fuller and Elmore do not qualify their conclusions 
with any reference to “sharp selection effects.” Yet 
the study by Wisconsin Professor John Witte, in- 
cluded in their report and upon which the quoted 
conclusion depends, fails to take into account the 
particularly disadvantaged character of the student 
population participating in the Milwaukee pro- 
gram. 

The Witte findings upon which Fuller and 
Elmore depend did not take into account parental 
education, parental occupation, welfare depen- 
dency, whether the household is headed by one or 
two parents, a student’s native language, and 
whether a student has severe social problems. The 
control for family income is woefully imprecise. 

Before entering choice schools, students had 
lower test scores and greater problems with public 
school than those who remained in public schools. 
They were more likely to be from particularly low- 
income families headed by a single parent. Their 
families were especially likely to be welfare depen- 
dent. They probably came from homes in which 
English was less likely to have been spoken. 
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racial isolation, to inner-city Hispanics looking for ing tales. We have the crucial and painstaking task of 

safety and a sense of control over how their children carefully determining which educational policies work, 

grow up. which fail to touch parents and kids, and why. 

The political battle will persist. Yet university-based 

scholars and professional educators truly interested For more information on Who Chooses, Who 

in the human effects of choice should tirelessly press for Loses? Culture, Institutions, and the Unequal Effects 
clear and carefully analyzed evidence. Let the politicians of School Choice (Teachers College Press , New York), 

argue about political rights and weave their intrigu- fax Professor Fuller at (617) 496-3095. 
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Interpreting the Harvard “Study” on School Choice 



By Terry 

Bruce Fuller is a reasonable guy whose professional 
values are in the right place. He believes in his own 
research — which, in point of fact, is fairly sympathetic 
to choice — and he feels that a one-sided blast by choice 
advocates does not do justice to the Harvard volume 
he has recently edited with Richard Elmore. I under- 
stand his frustration. But I also think the critics make 
some important points and that Fuller’s response does 
not take them seriously enough. 

First, some background. The Harvard volume was 
unveiled to the media with much fanfare, accompa- 
nied by a news release — a summary that was surely 
either written or approved by the editors themselves — 
which clearly indicated that the basic thrust of the 
study was negative. Choice is popular, they found, but 
it leads to social inequalities, fails to yield consistent 
gains in learning, and should be regarded with skepti- 
cism by policymakers. Major newspapers around the 
country soon followed with high-profile stories tell- 
ing the American public that, according to an impor- 
tant new study out of Harvard, school choice does 
not work and promotes serious problems. 

Pioneer Institute responded by gathering a panel 
of pro-choice experts to offer comments on the 
Harvard volume. The purpose was not to launch a 
probing, comprehensive evaluation of the details of 
each empirical chapter, nor to provide a representa- 
tive cross section of academic reactions to the book. 
The purpose was simply to set the record straight and, 
in so doing, to encourage a more balanced and better 
informed debate about choice. 

I will not try to cover all the issues here, but sev- 
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Abe editors try to pull these disparate 
efforts together by suggesting that they 
point to a reasonably coherent and es- 
sentially negative perspective on choice. 
But the evidence contained here could 
easily, and with every justification, be 
interpreted much more positively. 



eral matters deserve to be highlighted. 

(1) The Harvard volume does indeed emphasize 
the negative aspects of choice and downplay the posi- 
tive. The negative spin is quite apparent in the edi- 
tors’ news release, which introduced their book to the 
public. It is also apparent in their summary chapter, 
co-authored by Elmore, who is an open critic of choice. 
And it is apparent in the title the editors consciously 
adopted for their new book: Who Chooses, Who 
Loses ? Culture, Institutions, and the Unequal Effects 
of School Choice. Fuller should not be surprised at 
the reaction of choice proponents. Their perception 
of the book’s themes and packaging is accurate. 

(2) The Harvard “study” is not really a study in the 
usual sense of the term, and should not be presented 
as such. A study is a coordinated research effort in 
which the various components are interdependent and 
consciously designed to promote a systematic investi- 
gation of a particular problem. In this case, the edi 
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11 his, I think, is the flat-out truth of the 
matter: current choice programs do some- 
times lead to inequities, but these prob- 
lems are rooted in bad designs and can 
be mitigated through better ones. 



tors simply arranged a conference by inviting research- 
ers who were already involved in their own projects; 
these people came and presented papers on their sepa- 
rate projects; and the papers were collected into one 
volume. The volume is just the proceedings of the con- 
ference, with an introduction and conclusion added 
by the editors. The public relations blitz out of Harvard 
labeled this exercise a “two year long research project” 
and said that Fuller and Elmore “coordinated the re- 
search team.” This is inflated and misleading. 

(3) The outcome of this “study” could have been 
predicted with fair accuracy before the conference was 
even held. The reason is that the most visible, most 
widely influential invited participants were known to 
be critical of choice. Much of their research was al- 
ready underway (or finished) and known to empha- 
size negative themes. Would anyone, for instance, ex- 
pect Valerie Lee to write an article that reflects posi- 
tively on choice? What about Amy Stuart Wells, Jef- 
frey Henig, or John Witte who, while more balanced 
in their analyses than Lee, have all spent years map- 
ping out their criticisms of choice-based reforms? A 
conference built around these people is unlikely to yield 
any surprises, or to impress choice proponents as a 
dispassionate “research project” whose conclusions 
are driven purely by new evidence. 

(4) I have read every chapter of the Harvard vol- 
ume. They have little in common, and any serious ef- 
fort to evaluate their quality or significance would have 
to be done quite separately for each chapter. The edi- 
tors try to pull these disparate efforts together by sug- 
gesting that they point to a reasonably coherent and 
essentially negative perspective on choice. But the evi- 
dence contained here could easily, and with every jus- 
tification, be interpreted much more positively. In par- 
ticular, the editors (and chapter authors) could have 
noted that their main reservation about choice — the 



inequities that result when better-educated parents 
(even among the poor) take greater advantage of choice 
opportunities than other parents do — arise from prob- 
lems in the way these programs are designed, and could 
be dealt with effectively through appropriate changes 
in design. This, I think, is the flat-out truth of the 
matter: current choice programs do sometimes lead 
to inequities, but these problems are rooted in bad 
designs and can be mitigated through better ones. In- 
terestingly enough, the editors and authors actually 
try to stress the importance of design. But they do not 
follow through on this. And for some reason they feel 
compelled to treat inequities as permanent and damn- 
ing properties of choice systems, rather than going 
ahead to argue, as their own logic would seem to re- 
quire, that good designs can go a long way toward 
eliminating these problems. 

In conclusion, I should simply emphasize that the 
“facts” marshalled by these studies are not in them- 
selves troubling for choice and its supporters. There 
are clearly important advantages to choice, and there 
are clearly problems associated with existing programs. 
For the most part, the Harvard volume recognizes 
specific advantages and problems that I can agree with. 
On the brute facts, I do not think we differ too much. 



On the brute facts, I do not think we dif- 
fer too much. We differ on the interpreta- 
tion. Theirs is highly negative, and in my 
view there is no good justification for it. 



We differ on the interpretation. Theirs is highly nega- 
tive, and in my view there is no good justification for it. 

In both politics and social science, it is the inter- 
pretation that ultimately has influence, not the brute 
facts. So it is important that these issues of interpreta- 
tion be discussed and debated, and that all of us try to 
work our way toward the truth, whatever it may be. 
Pioneer Institute’s critique of the Harvard volume, 
Fuller’s response, and my reaction to Fuller are all le- 
gitimate and useful parts of that process. With any 
luck, there will be much more to come. 
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